It has become difficult to ensure a sufficient amount of exposure at the time of imaging along with an increase in speed of cameras and miniaturization of imaging elements. As a result, a captured image becomes a very noisy image. Here, the noise occurred in the image can be removed by a signal processing. A noise removal processing method can be classified roughly into a spatial filter processing and a temporal noise removal processing.
When the noise is removed by a spatial filter, there are problems such as a tendency for an edge of an image to be blurred and a decrease in saturation. On the other hand, the temporal noise removal processing makes it possible to suppress occurrence of the problems when the noise is removed by the spatial filter, while effectively removing noise.
As a conventional temporal noise removal method, there is a method in which noise is removed by performing spatial motion compensation on images in a time direction and then adding each of pixels in an input image and a corresponding one of pixels obtained by the motion compensation.
FIG. 9 is a functional block diagram of a conventional image processing device that performs the temporal noise removal processing.
As shown in FIG. 9, a conventional image processing device 20 includes a motion estimation unit 11, a frame memory 12, a motion compensation unit 13, an addition unit 14, and an addition ratio calculation unit 16.
First, the motion estimation unit 11 performs motion estimation using an input image and a reference image that is stored in the frame memory 12, and determines locations, each of which is associated with a corresponding one of blocks in the input image, in the reference image.
Next, the motion compensation unit 13 generates block images of the reference image which are at the locations each associated with the corresponding one of the blocks in the input image, according to the motion information determined by the motion estimation unit 11. Then, the addition unit 14 adds the motion-compensated reference image and the input image on a per pixel basis. This addition processing reduces noise included in the input image by temporally smoothing random noise.
Here, the addition ratio calculation unit 16 calculates an addition ratio between the input image and the motion-compensated reference image. The addition unit 14 adds the input image and the motion-compensated reference image using the addition ratio calculated by the addition ratio calculation unit 16.
The addition processing described in Patent Literature 1 is described as a processing to be performed by the addition unit 14.
FIG. 10 is a diagram for describing a pixel addition method performed by the conventional image processing device.
In the image processing device 20 shown in FIG. 10, the motion estimation unit 11 detects a motion vector (MV) using the input image and the reference image stored in the frame memory 12.
Next, the motion compensation unit 13 generates a motion-compensated reference image through motion compensation in which the motion vector is used. Furthermore, the addition ratio calculation unit 16 calculates a coefficient α using a value of MAD (Mean Absolute Deviation) calculated by the motion estimation unit 11. The MAD is an amount indicating an error between the two images. For example, MAD between an image 1 and an image 2 is defined by (Equation 1).
      [          Math      .                          ⁢      1        ]                                MAD          =                                    1              N                        ⁢                                          ∑                                  i                  =                  1                                N                            ⁢                                                                                    f                    ⁢                                                                                  ⁢                    1                    ⁢                                          (                      i                      )                                                        -                                      f                    ⁢                                                                                  ⁢                    2                    ⁢                                          (                      i                      )                                                                                                                                                (                      Equation            ⁢                                                  ⁢            1                    )                    
Here, f1(i) and f2(i) are pixel values of the images 1 and 2, respectively. N is the number of pixels.
Here, when MAD between a block of the input image and a block of the reference block which corresponds to the block is small, it means that a difference between the block of the input image and the block of the reference image is small. To put it differently, in this case, it is highly likely that the motion estimation has been performed correctly.
The coefficient α indicating an addition ratio of the reference image is increased by using such characteristics of the MAD when the MAD is small, and a noise removal effect is enhanced by increasing the addition ratio of the reference image.
On the other hand, since it is likely that the motion estimation has not been performed correctly when the MAD is large, the coefficient α is decreased, and control is performed so that the addition ratio of the reference image is decreased.
Moreover, in the noise removal processing, an image is divided into blocks and processed on a per block basis. Here, an addition ratio determined is an addition ratio per block. However, the addition unit 14 requires an addition ratio per pixel because the addition unit 14 performs addition on a per pixel basis. Thus, the conventional image processing device 20 calculates the addition ratio per pixel by interpolation, using addition ratios of neighboring blocks.
[Citation List]
[Patent Literature]
[PTL 1] Japanese Unexamined Patent Application Publication No. 11-504490